A fflGH PERFORMANCE GRAY-LEVEL WATERMARKING 
SYSTEM 



FIELD OF THE INVENTION 

The present invention relates to the watermarking system for data 
protection. More particularly, the new invention system can provide 
gray level signal watermarking for images, video, music and speech 
applications. 

BACKGROUND OF THE INVENTION 

In recent years, Internet system has become popular for transmitting 
and receiving digital data. Because the nature of Internet is an open 
system, transmitted data is easily copied, altered or even stolen. Hence 
communication security becomes a major concern. Moreover, digitized 
multi-media data such as image, video and music signals have been 
widely transmitted using this open system for its convenience. The 
problem of copyright protection becomes a most controversial subject. 
Unfortunately, at the present time, the protection technique is nowhere 
near perfect. Hence large amount of unauthorized duplication is found 
daily. In order to resolve this problem, watermarking techniques [1-4] 
have become a popular tool to protect the secrecy of the transmitted 
document. To achieve a high performance watermarking, the algorithm 



must possess properties of invisibility, security and robustness. In recent 

literatures, most authors [5-7] concentrated on using binar>' date 
watemiarking, such as "logo" or "text", the gray-level image 
watermarking for protecting muiti-media data has rarely been discussed. 
5 In this invention, a gray-level data watermarking based on sub-band 
filtering and codebook principles is developed. In order to accomplish a 
high security watermarking, we avoid using the original data to extract 
watermark. Instead, we develop a private-key system for protecting 
watermark, which gives an additional security level. 

10 

SUMMARY OF THE INVENTION 

Although many watermarking algorithms have been developed in 
recent years, most of them concentrated on using binary data such as 

15 logos or texts. In this invention, the sub-band filtering approach is 
successfully employed for a gray-level data watermarking. With signal 
processing in the firequency domain, the feature of progressive 
transformation is utilized to disperse the watermark into the entire data 
and to enhance the robustness. The watermark is randomly dispersed into 

20 the high fi-equency portion of the original data. We achieved good hiding 
quality, which makes the unfriendly attempt on either duplication or 
alternation extremely diflScult. To extract the watermark, we develop a 
four-fold key system rather than using the original data. And the privacy 
key is fiirther encrypted to increase the degree of security. Furthermore, 
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the progressive feature of DCT is used to enhance the robustoess of 
watermarking. Even if the composite data is under attacked, the 
watermark can be restored from the residual coefficients, where the 
resulting watermark only suffers different degree of blurring but no 
serious distortions. Our watermark algorithm possesses the nature of good 
vision quality, high security, and a strong robustness. The watermarking 
scheme can provide good protection on the secrecy of multi-media data 
transmission over the current Internet system, and no double shall 
contribute substantially on resolving the copyright violation problems. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing aspects and many of the attendant advantages of this 
invention will become more readily appreciated as the same becomes 
better understood by reference to the following detailed description, 
when taken in conjunction with the accompanying drawings, wherein; 

Fig. 1 is a flow chart of the embedding process for watermark based 
on sub-band theory. 

Fig.2 is a flow chart of the watermark extraction. 

Fig. 3 is a flow chart of the watermark embedding process using 
codebook concept. 

Fig. 4 a flow chart of the encoder and the decoder using the same key. 

Fig. 5 is a flow chart of the codebook method for watermark 



extraction. 

Fig. 6 is a flow chart of the mixing system by encryption and 
watermarking methods. 

Fig. 7 a flow chart of the second layer processing using encryption 
and spatial domain hidden. 

Fig. 8 is a flow chart of the watermark data extracting jfrom two 
layei^ hidden. 

Fig. 9 is a flow chart of embedding the privacy key into JPEG 
domain. 

Fig. 10 is a flow chart of the watermaiic data being extracted from 
JPEG file. 

Fig. 11 shows privacy keys sequentially inserted into non-zero 
coefficients. 

Fig, 12 shows the key insertion with the zigzag scan for LL band. 
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

It is well known that the spectrum of a data can be divided into four 

sub-bands [8], the LL band, LH band, HL band and HH band to 
represent its frequency components of 2D-signal. Filters for different 
bands can be obtained from the following equations 

LLband:XLL-=Hu,(xtj^=^^i: fx.W^'^Wi'^, (1) 

^ »1=0 b2=0 

UIbcmd:Xui=HLH(x^^^t 2x^^*"^5P;-*^^«^ (2) 

^ Bl=0 «2=0 



^ rel=0 n2=0 

HHband:XHH=HHH(Xij)=j;^ 2^j*^r**'^'f^2"^^'^"'; (4) 
where C is a constant for trmisformation. Using different frequency 
factors Wl and W2, transformations such as the Discrete Fourier 
Transform (DFT), Discrete Cosine Transform (DCTX etc. can be 
selected at will We adapt the DCT for it has been widely used in 
watermarking applications. In order to reconstruct the data without 
frequency aliasing, transformation should have the Quadrature Mirror 
Filter (QMF) feature [9]. To confirm QMF in 2D transforms, 
horizontal symmetry, vertical symmetry and cross symmetry properties 
must be achieved jSrst. In discrete signal processing, a 2D spatial block 
(0k} is transformed into a DCT block by 

^uv - Cay Z S fjk f%=^^CT(fj, ; , 4^ = cosi^^^) co^^^^X -(5) 

DCT transformation can easily achieve horizontal symmetry, vertical 
symmetry and cross symmetry by observing W^_^_^^ = (-1)"^^ , 

W^-j-m=ir'^TW^, aJid ^^c^-.-ix/.-*-i) = (-l)""f^J respectively For 2D 
sub-bands, we can define each band according to equations (1H4) as: (a) 
LL band Fx£=i^Mi,vi. |ul!<N/2 and |vl|<N/2; (b) LH bandF£g=F«2,v2. 
N/2> \u2\ and N>\v2\m2-, (c) HL band FhiT Fu3^v3> N>\u3\^/2 and 
\v3\<N/2; (d) HH FhiT Fu4y4, N>\u4\m2 andiSr>|v¥|>iV/2. 



In order not to confuse the "watarmark data" and "watermariced 

data", we shall name the former as the "watermark" and the latter as the 
"composite data" hereafter. Let /^denote a composite data where a 

watermark Wj^ has been embedded onto an original datajj^ Clearly, the 
nature of a watermark is that it must be perceptually invisible in order to 
prevent any attempt on the composite data by unfriendly sources. 
Intuitively, the high frequency portion of the original data is an ideal 
place to insert a watermaik since the great variation in high frequency 
makes the watermark much more difficult to detect. As the original data 
is dividGd into 8x8 blocks, its sub-bands can be obtained from 

The HH band components i^can be viewed as a measuring parameter 
to search for high frequency blocks. Let us define Phh as 

PHH=X\^^^ih^u.M\ > (7) 

which denotes a high frequency fector of the HH sub-band. As the Pg^ 
value is larger, the block frequency variance becomes higher. We search 
and record the Pff^ value of each block in accordance witli its relative 
block coordinate (m,n), and use tiie Phh (mn) to denote the high 
frequency variance for (m,nf' block. If the Phh (m,n)> 
Variance-Threshold, which means tiiis block is capable to embed a 
watermark without visible distortion. Then tiiis high frequency block 
should be mariced, for next discussions, let M(m,n) denote a coordinate 
of one marked block. 



In order to disperse the spatial correlation of the watermark, the 
entire vvatennark Wj^ is transformed by DCT. Then the DCT 
coefficients can be split into sub-bands as 



W^=FDCTiw.)^ 



where the watermaric size is mlxm2. Since the Wn band contains more 
information about the watermark, only tlieir DCT coefficients are 
embedded into the marked blocks. The value of the LL band coefficient 
is usually large due to the foil picture transformation. To reduce visible 
distortion, we need to quantize these coelBGcients before the embedding 
operation. Then equation (8) is simplified to 



(9) 



where Qmiin2 are parameters from a quantization table. Note that the rank 
of is one quarter of that of . In order to reduce the bit-rate of 
the watermark fiirther, we repeat the above proc^ure to produce a 
second layer LL band from the first layer W^^. Thus total bit-rate 
of the watermark is now reduced to 1/16 of the original one. 

As coefficients of are embedded into the marked blocks, we 
aim to achieve good hiding quality, high security and strong robustness. 
This can be easily done since only one coefficient of W^^ is inserted 
into the high frequency portion of each marked block. Thus the 



watermarked block becomes 

^-~\Fn. F, 

where contains one watermaiic coeflScient. Note that the marked 
blocks are randomly distributed, hence the watermark information is 
dispersed on the original picture everywhere. Finally, DCT coefiBcients 
are converted into spatial domain by using the inverse DCT 
transformation, hence the composite data is obtained by fj^=IDCT(F^) 
of each sub-block. 

The marked coordinate is an important information for restoring the 
watermark^ M(mn) can be treated as a key to restore the watermark. Let 
us define a privacy key PK as 

PK={M1M2, ... ,Mnl MxczMfmn), x=l to n, (11) 
which is a set of coordinates of the marked blocks. In order to attain 
high security, the privacy key can be further re-pennuted by using 
conventional coding techniques, such as data encryption, etc. [10]. 
However, no algorithm is capable to remain undecoded forever, but we 
can cr^te multiple privacy keys for our watermark with time-dependent 
codes to increase the difficulty of decoding and hence to prolong the 
duration of secrecy. Therefore users have a Key Bank, KB^{PKO, 
PKl,..., Y}, where KB contains Y privacy keys to be used at various 
appropriate times. The complete embedding procedure of our algorithm 
is illustrated in Fig. I . 

In our watermarking system, we employ the designated keys rather 




than tiie original data to extract watermark. These keys are computed 
according to the frequency variance of original datas, and are 
independent of the watermark. So they can be pre-stored on disks in the 
decoder of the ajuthorized user. To extract watermark, we only need to 
5 send the composite data to users without Ihe keys. The key in our 
algorithm has four folds. They are (a) Privacy Key: provides the 
position of the marked blocks, (b) Permutation Function key: decrypts 

random values into normal ones, (c) Quantization Table of JF^^ : 
provides the de-quantization for DCT coefficients of the watermark, (d) 

10 Embedding Coefficient Location: extracts the watermark coefficient 
from HH band of the marked blocks. In general, the procedure of 
e^raction is just the opposite of the embedding operation. The keys that 
are pre-stored in tiie Key Bank use the same parameter for both encoder 
and decoder, wherc each user key is corresponding to one original data. 

15 Since the composite data is very similar as the original data, we can 
select one user key from Key Bank according to the composite data 
without any additional information required. 

Fig. 2 shows the steps for watermark extraction. When users receive 
a composite data, it is first partitioned into 8x8 blocks. Since the 

20 watermark is hidden on DCT domain, these blocks should be 
transformed by DCT first, i.e. F^{Extract)=DCT{f ^)sx8- As previously 
mentioned, watermark coefficients are embedded in the high frequency 
blocks, and the position of these marked block is recorded in the privacy 
key. By referring the different composite data, one can choose the 
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appropriate privacy key from the Key Bank where the key order is 
re-permuted by decryption procedures. Hence the watermark 
coefficients are sequentially extracted from the HH band of {Extract) 
in the marked blocks. Finally, the LL band matrix of the watermark can 
be r^x)fistructed. To restore tiie waterm^k, we need to combine the LH 
band, HL band and HH band of the watermark for a complete 2D 
sub-band processing. However, data in these bands are not used in our 
approach, so we must insert zeros to LH, HL and HH bands to achieve a 
complete matrix. After a de-quantization procedure, we can reconstruct 
the coeflScient matrix of the watermark, and finally the watermark 
can be extracted from the inverse DCT ot . 

The other watermarking invention uses the codebook concept. In 
the first stage, the original data and watermark data all need be 
transformed to DCT domain. When the frame size of original picture 
fjk is MlxNl, we take a full-frame transformation. Thus the DCT 
coe:^cient^ are attained form 

F 



FuJvl ~FI>CT(f^Ml>cNl = 



(12) 



where ul and vl are frequency indexes for coefficient Fuiyi By the 
same way, we attain the watermark coefScients from 
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(13) 



where the watermark daia size is M2><N2, and its frequency indexes use 
u2 and v2. To avoid the visible distortion after embedding process, the 
high frequency part of F^iyi is good choice to insert watermaiidng 
information since human eyes are difiScult to detect high frequency 
component. However, when the watermaric coefficient is embedded 
to the high frequency part, the watermark coefScient should be lost after 
data compression because high frequency component is spatial 
redundancy. 

Based on frequency codebook concept, we propose a new approach 
to realize the watermarking system. As a watermarking coefiScient Wa 
is to be embedded, its vector can be found from codebook. The 
coefiScient matrix in (1) is treated as a codebook content. So the 
codebook is a set of picture coefficients. With coef5cient-by-coefficient 
searching, we can find the minimum differential value between 
watermarking coefficient and codebook coefficient after nlxml points 
compared, which is given by 

(Dmmin=Mm{Wa-Fj,l Fi^eF^j^j, (14) 

where Min is minimum value search fimction. The watermarking 
coefficient is matched the codebook coefficient by minimum difference. 
The relative coordinate in matrix (1) can be recorded as a vector. To 
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embed the watermarking information, the watermarking coefficient is 

ratiier than the originai picture coefficient with 

Wb]eim(xnmr^a, (15) 

where [F&]ei>i;^inin)is corresponding to the vector location of minimum 
difference. By the same way, the next watermarking coefficient is 
processed. The relative coordinate also requires to be recorded for each 
coefficient. Anyway, when the codebook coefficient has replaced by one 
watermarking coefficient in the previous search, th^^ coefficient is not 
changed again for the next watermark coefficient processing. Totally, 
there are n2xm2 watermaiking coefficients to be embedded. Finally; 
we can attain the relative vector according to the coordinate location 
from each watermark coefficient. Hereafter, the vector is named as 
Wa(m,n) to denote the coefficient Wa. located on {m,n) coordinate. The 
Wa(mn) is an important information to restore watermark. So Wa(m,n) 
can be treated as a privacy key (PK) [22] that is defined by 

PK={MlM2...Mn}, Mxcz Wa(m,n), x=l to nixml, (16) 
PK is a. set of vector for watermarking information. 

After the embedding process, the transform coefficient of original 
pictere involves the watermarking coefficient. We get the new 
coefficient matrix with 



IF W F F W F 

' ^^^^ 

where some coefficients have been replaced by watermarking 



02 03 '^'^12 ^ 05 • 

J^2i J^^si^ia ^13^^14 K5. . 

^20^^05^6Z^^33^24^02 



12 



coefficients. Finall>; the frequency coefficients are converted into spatial 
domain using inverse transform with 

fi-IDCT(F^,^,), (18) 
/J denotes the composite data in the first layer output. The embedding 
procedure of the proposed algorithm is illustrated in Fig. 3. 

The procedure of extracting operation is an inverse operation of the 
embedding one. As receiving the composite data, we only need the key 
parameter rather than original data to extract watennark. Since the key 
is a secret data, we need use ihe same key both of the transmitter and 
receiver, as shown in Fig.4. Fig. 5 indicates the processing step for 
watermark restoration. When the composite data is received, the data 
should be transformed by DCT since the watermarking information is 
hidden on the frequency domain. So we have 

F^=i?crr7;),i,^i. (19) 

Using the privacy key, we could fmd the watermark coefficient from 
codebook. Then each coefficient is sequentially extracted according to 
the privacy key and the codebook content. Then the watermark 
coefficient matrix can be reconstructed. The reconstructed 
watermark data could be attain from inverse transform of with 

<^^ = IDCT(W^)^2.m2 (20) 

The wateraiarking information is hidden into the composite data 
and its privacy key. In order to increase security, we propose the 
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second layer processing to hide tlie privacy key to enhance the 
communication security. Since we have used the frequency domain 
processing in the first layer, in order to avoid the frequency spectrum 
aliasing, we employ spatial domain processing consisting of the 
encryption approach in the second layer. 

The encryption approaches had been used in the past twenty years. 
As the secret data is enciypted, the output becomes a scrambled data by 
predetermined key. The attacker can easily detect the data whether or 
not is encrypted. Since the computing power increases very fast today, 
the complex encryption system also can be de-encrypted by using 
statistical analysis. Although the watermarking system can efficiently 
avoid the attackers' detection, the security level is low. Once the 
watermark is detected, the attacker can be easy to extract the secret data. 
Therefore, we develop a mixing system with the encryption consisting 
of watermark system. The b^ic structure is shown in Fig. 6. The 
encryption can increase the security level for watermark. And 
watermarking can protect the encryption to be detected. 

In our watermarking system, the privacy key is treated as a secret 
information. In order to increase the security level, the privacy key can 
be re-permuted by using conventional encryption methods, such as DES. 
As the key is encrypted by the encryption method, we have the new PK 
(NPK) from 

NPK^ Encryp (PK). (2 1 ) 

The enciyption approach can provide high security to protect the 
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privacy key. In order to avoid the encryption operation to be detected, 
the second layer hidden is employed. 

To enhance securit>; the new PK can be further hidden into the 
composite data using spatial domain processing to avoid attackers' 
detection. One has simple and efficient method, the key is considered 
to embed the LSB bit of pixel in composite data, then we have 

fi={f;,)LSB^iNPK)M , (22) 
where f^is the second layer composite data. The NPK data use 
bit-by-bit to insert the LSB of from left-to-right and top-to-bottom 
sequentially. The first layer consists of the second layer with encryption 
and spatial domain processing is shown in Fig. 7. The decoding system 
block is illustrated in Fig. 8. As we receive fl, the LSB is extracted to 
attain the key information. After de-encryption, the privacy key can be 
re-restored. According to the privacy key and f^, we can reconstruct 
the watermark data. 

As the composite data size is mlxnl, we have mlxnl bits space to 
embed the key information. The coordinate resolution (m,n) presents 
each watermark coeflBcient vector, then the watermarking data size 
m2xn2 can be determined by 

mlxnl. (23) 

m + n 

As the key information is embedded into LSB bits of each pixel, the 
data quality degrades very small. Moreover, the privacy key has been 

., 15 , , , 



encrypted, to detect the watemiarking information becomes very 
difficult. Thus the security level becomes higher as double hidden 
procedure is used. In fact, this watermarking system is the mixing 
process of the frequency and spatial domain hidden. In first layer, the 
j&^uency codebook watermark is employed using the DCT domain. 
And the privacy key is scrambled using the encryption approach 
consisting of spatial processing in the second layer. 

Currently, JPEG compressor has been widely used to reduce the 
data size. However, most of tiie watermark data is discarded after JPEG 
processing as well. In order to overcome this drawback, a high 
robustness algorithm for JPEG processing is invented. In the JPEG 
compressor [12], the data is partitioned into 8x8, and then is 
transformed by DCT processing. These DCT coefScients are quantized 
using a particular quantization table, which can be expressed as 

4 = ^^. (24) 

where 0^ is the quantization level that is dependent on the u and v 
component, is a composite data that is attained from (8), and 
is the final quantized coefficient. For RLCAn^^C package, the coefiScient 
should be truncated into an integer. In feet, the LSB bit of each 
coefiQcient is not exact after the truncation processing. Thus the 
compression performance is not changed too much as the privacy key 
information is embedded into LSBs of non-zero coefficients. With this 
concept, we first search all non-zero coefficients. Then the LSB of 

...16. ..... . . . . 



non-zero coefiQcient is modified by one-bit of the privacy key, wMch is 

given by 



W (4) is^en,ihsii iF^)^s=i,asiKey)^,^=l , asF^^O (25) 

[Else (F^.)^ Mo Change 

The information of the privacy key is sequentially inserted into the LSB 
location of each non-zero coefiQcient with bit-by-bit approach. Since 
only LSBs of non-zero IXIT coefScients are modified, the compression 
ratio is not affected. 

Fig. 9 illustrates the privacy key embedded into non-zero 
coefficients on the JPEG encoder. Fig. 10 illustrates the JPEG decoder 
system block for the watermark data extracting. From the JPEG file, 
the composite data is reconstructed after JPEG decoder. Simultaneously, 
the privacy key is extracted fi-om each non-zero coefficient. Then the 
composite data is transformed into DCT domain. According to the 
privacy key, the coefficients of the watermark data can be extracted. 
Finally, the watermark data can be restored from inverse DCT 
processing. 

From the above mentions, how much the key information embedded 
has to be counted. The total bit of the key infonnation is 
n2xm2xiQg^(raxmi) since there are n2xm2 keys and each key with 
1<^2(h1xot1) bits. After JPEG processing, the number of non-zero 
coefScients (bJNZC) in the full frame is computed by 
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If F^^O,-ais&NNZC = NNZC + l, where i = lto(nl/H)x(ml/S), u,v = 0to7 
(26) 

, where denote the quantized coefficient at the (tt,v) location for the 

f block, and the composite data is spht into (nl/8)x(ml/8) blocks as a 
block size is 8x8. To embed the key information completely, the 
following equation 

NNZC ^nlxmlxlog^irAxml) , (27) 
need be satisfied. Otherwise, some key information would be discarded, 
and tiie extracting quality will degrade accordingly. In fact, the NNZC 
becomes smaller after quantization. Eq. (27) is seldom satisfied in 
practical case, except the watermark data size is fiirther reduced or the 
source data size is further enlarged, but the hiding system shall become 
no efSciency. For practical applications, we present three scanning 
approaches as below. 

Since DCT transformation has a progressive feature, each 
coefficient implies one spatial resolution. The watermark data can be 
restored using only a few DCT coefficients. The sequential procedure is 
shown in Fig. 11, where Uie key information is scanned to insert JPEG 
bit-stream with block-by-block processing. Until all non-zero 
coefficients are completely processed, the inserting procedure is stopped. 
Because the amount of the key information is much larger than that of 
non-zero coefficients, the extracting quality is not good generally. 
Intuitively, we can increase the number of non-zero coefficients to 



improve the data quality, but this scheme will reduce the compression 
ratio. Instead, the high-efficiency scanning styles are presented. 

Because most of the data energy is compacted into the low 
frequency band in the DCT domain, the key information located at the 
low frequency area is first considered to embed. Based on a sub-band 
concept the coefficients can be split into LL, LH, HL and HH bands. 
In order to keep a regular processing flow, only LL band information is 
imbedded by using the zigzag scan as shown in Fig. 12. The zigzag 
scan likes as the run length coding within JPEG compressor, the 
scanning direction is from left-top comer to right-bottom for the LL 
band only. 

Since the DCT coefficients are independent, the data can be 
apiMX)ximately reconstructed from inverse transformation with 
coefficient-by-coefFicient approach. As the coefilcient value is larger, its 
weight is larger to affect the reconstructed result. To improve the data 
qualify, the key position corresponding to a larger coefficient is first 
inserted. Because the coefficients are random distribution on the 
codebook, one-bit flag is required to record the current coefficient 
whether or not to be selected. For LL band processing, there are 
n2/2xm2/2 bits used to mark the available coordinate. The number of 
maximum weight coefficient (NMWC) recorded for the key processing 
can be computed as 

log2(«l + M) ' ^ ^ 

The processing procedures using the maximum weight scan are: 
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(a) . Pre-computing NNZC and NMWC. 

(b) . Find the maximum coefficient and record its corresponding key 
position. 

(c) The one-bit flag is set to high, and counter=counter+l . 

(d) The next maximum coeflScient is processed with repeating 

(bHc), 

until the counter =iVM^'C. 
Although we sacrifice one bit space to record the location of maximum 
weight coefficients, the larger weight coefficients used can improve the 
inverse transformation result 

As previously mentioned, we had used two-layer DCT domain 
hidden. First the DCT coeflBcients are hidden on the fi-equency 
codelx>ok. The codelxjok vector is treated as a privacy key. Then the 
privacy key is further hidden into DCT coefficients in the bit stream of 
JPEG domain. This approach can meet three requirements for the data 
hiding systems: invisible, robustness and security. The watermark data 
becomes invisible since its coefficients are embedded with the best 
match from the codebook content. Our approach can provide a high 
robustness for JPEG compressing because of the random coefficient 
distribution over the entire data. With double DCT layer hidden, a high 
security for the watermark data protection is achieved. 
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